Multiparty multimodal interaction: a preliminary analysis

When people work together, they often talk about the objects in their environment. Not surprisingly, their dialogues are multimodal, incorporating speech, gesture, gaze, haptics, and perhaps other modalities. However, proponents of technology may be troubled to learn that despite the current state and future promise of spoken and multimodal research, many of these workers do not particularly want to talk to machines – they want to converse with their colleagues. Still, if there were unobtrusive computer support for their multimodal dialogues, these same individuals would be pleased to benefit from digital technology. This paper offers a first step towards building such multimodal systems for supporting face-to-face collaborative work by providing both qualitative and quantitative analyses of multiparty multimodal dialogues in a field setting.